Alerting system for automatically detecting, categorizing, and locating animals using computer aided image comparisons

ABSTRACT

Described are systems for (a) monitoring real-time animal activity in an area of interest using computer aided image comparison of a real-time image of the area and reference images of animals of interest and (b) providing informed alerts to a user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit and priority of U.S. Prov. Pat. App.Ser. No. 62/027,965 (filed Jul. 23, 2014). That document is herebyincorporated by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

BACKGROUND OF THE INVENTION

1. Field of Invention

The disclosed subject matter is in the field of systems and apparatusfor (a) automatically detecting, categorizing, and locating animalsusing computer aided image comparisons and (b) alerting a user to thedetection of a preselected animal or activity of said preselected animalin an area of interest.

2. Background of the Invention

Sometimes, animal activity in a particular area of interest (“area ofinterest”) is monitored. In particular, many are desirous of knowing theanimal activity in an area of interest so that they can be alerted tothe presence of a specific animal in the area or to a specific type ofanimal activity in an area. Game hunters, for instance, are frequentlyinterested in game animals' presence around a lure (e.g., a deer feeder)so that the likelihood of a successful hunt is increased. Similarly,animal enthusiasts or scientists are interested in the presence ofparticular animals in an area for observational purposes. Finally,farmers monitor their crop and livestock areas for dangerous ordestructive animals so that undesirable animal behaviors can beprevented or deterred.

In view of the foregoing, cameras have sometimes been used to record theanimal activities of an area. However, such cameras are not alwayssuitable for monitoring an area of interest. Reviewing camera images orvideos in real-time is time consuming, and after-the-fact review of suchimages and videos results in delayed alerts of animal activities.Techniques like motion detection and time lapse photography have beenemployed to minimize the volume of real-time images or video to beanalyzed while monitoring animal activity in an area of interest.However, these techniques are indiscriminate with respect to the typesof animals that are detected, and frequently result in falsealarms/alerts or the reporting of irrelevant animals or animalactivities (e.g., a squirrel, or even wind, can cause the same alert asa game animal or a mountain lion).

It comes as no surprise that a need exists for improved systems andapparatus for (a) automatically detecting, categorizing, and locatinganimals using computer aided image comparisons and (b) alerting a userto the detection of a preselected animal or activity of said preselectedanimal in an area of interest.

SUMMARY OF THE INVENTION

An object of this disclosure is to describe systems for: (a) monitoringreal-time animal activity in an area of interest using computer aidedimage comparison of a real-time image of the area and reference imagesof animals of interest, and (b) providing informed alerts to a user.Sometimes, the alerts can suggest potential responsive actions such asdeterrents to the detected animal or its behavior. In a preferredembodiment, the system consists of a camera and a computer operatedmicroprocessor mounted in a remote area of interest, e.g., in thevicinity of a game feeder. In another preferred embodiment, the systemconsists of a camera mounted in a remote area of interest, e.g., in thevicinity of a game feeder, connected wirelessly to a centrally locatedcomputer operated microprocessor. In one instance, the microprocessor isloaded with reference images of animals. In one mode of operation, thesystem “learns” the static background of the area of interest (i.e., theappearance of the area without any moving objects) while real-timeimages are continuously provided by the camera to the microprocessor.The program algorithms compare the real-time images of the area to thestatic reference background image to identify any areas of incongruitybetween the static image and the real-time images. Suitably, referenceimages of animals of interest are compared to the incongruities in thereal time images to determine whether the incongruity in the real timeimage is an animal of interest (e.g., by comparing the shape, size, andcolor of the incongruities with the animals in the reference images;image entropy, edge detection, and face detection algorithms might alsobe incorporated into the comparison protocols). If, after comparison,the incongruities in the real-time images have a match with thereference images, then a wireless signal alerts the user. If thedetected animal is relevant, the user is notified of the animal'spresence and activity in the area.

Other objectives and desires may become apparent to one of skill in theart after reading the below disclosure and viewing the associatedfigures. Also, these and other embodiments will become apparent from thedrawings.

BRIEF DESCRIPTION OF THE FIGURES

The manner in which these objectives and other desirable characteristicscan be obtained is explained in the following description and attachedfigures in which:

FIG. 1—is an environmental view of an area of interest, which viewillustrates a notional remote setup of the disclosed system;

FIG. 2—is a high level logic flowchart for the algorithms controllingthe system;

FIG. 3—is a detailed logic flowchart of the algorithms controlling thesystem.

FIG. 4A—is an illustrative view of the system;

FIG. 4B—is an illustrative view of the system;

FIG. 4C—is an illustrative view of the system;

FIG. 4D—is an illustrative view of the system; and,

FIG. 5—is an illustrative view of the system.

It is to be noted, however, that the appended figures illustrate onlytypical embodiments of the disclosed assemblies, and therefore, are notto be considered limiting of their scope, for the disclosed assembliesmay admit to other equally effective embodiments that will beappreciated by those reasonably skilled in the relevant arts. Also,figures are not necessarily made to scale.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Described are systems for (a) monitoring real-time animal activity in anarea of interest using computer aided image comparison of a real-timeimage of the area and reference images of animals of interest and (b)providing informed alerts to a user. In a preferred embodiment, thesystem consists of a camera and a computer operated microprocessormounted in a remote area of interest. In another preferred embodiment,the system consists of a camera mounted in a remote area of interest,wherein the camera is connected wirelessly to a computer operatedmicroprocessor. In one instance, the microprocessor is loaded withreference images of animals. In one mode of operation, the system“learns” the static background of the area of interest (i.e., theappearance of the area without any moving objects) while real-timeimages are continuously provided by the camera to the microprocessor.The program algorithms compare the real-time images of the area to thestatic reference background image to identify any incongruity in thereal-time images. Suitably, reference images of animals of interest arecompared to the incongruities in the real time image to determinewhether the incongruity in the real time image is an animal of interest(e.g., by comparing the shape, size, and color of the incongruities withthe animals in the reference images; image entropy, edge detection, andface detection algorithms might also be incorporated into the comparisonprotocols image entropy, edge detection, and face detection algorithmsmight also be incorporated into the comparison protocols). If, aftercomparison, the real-time images have a match with the reference images,then a wireless signal alerts the user. If the detected animal isrelevant, the user is notified of the animal's presence and activity inthe area.

FIG. 1 presents a notional description of the remote camera sensingsetup for the system 1000. Suitably, the system consists of a camera1100, microprocessor 1200, and wireless communication device 1300mounted in a remote area of interest, e.g., the area in the vicinity ofa deer feeder 2000. Suitably, the camera 1100 is configured to captureimages or videos of the area of interest and load those images to themicroprocessor. In a preferred embodiment, the microprocessor 1200 (a)is pre-loaded with references images of animals of interest, (b) iscontinually learning the appearance or background of the area in astatic state (i.e., without any animals or movement) by a constant photoor video feed from the camera (e.g., to account for snow melt, changesin lighting conditions (shadow movement), movement of the camera orfeeder), and (c) is configured to compare the images or video imagestaken by the camera with the reference images to determine the type,size, and color of animals entering the area of interest. A wirelesscommunication unit 1300 is provided for sending alerts regarding theanimal's information.

As alluded to above, the microprocessor 1200 features computer code inthe form of computer image comparison algorithms for comparing images orvideo from the camera with reference images of animals of interest. FIG.2 is a high level logic flowchart for the algorithms that control thisfunctionality. First, an image of the static state background orappearance of the area is learned. The learning of the static backgroundor appearance of the area is discussed in greater detail below withreference to FIG. 3. Second, reference images of animals of interest anda static image of the area of interest may be input and stored in theonboard computer microprocessor. In a preferred embodiment, forinstance, images of a male deer (“buck”) or female deer (“doe”) takenfrom different angles or orientations may be loaded on themicroprocessor. For instance, there may be images reflecting a buck in aleft-side orientation, a right side orientation, a front orientation, ora back orientation. Preferably, the reference image could be an emptyscene (i.e., no animals) in the area of interest). Suitably, thereference images could be of any animal. In the preferred embodiment,the reference images are designed to characterize a particular animal ina multitude of orientations. The number of reference images for aparticular animal or empty/static background image is not set, but alarger number of reference images would typically yield a higherpotential for a match between the reference images and the real-timecamera images.

Third, with the reference images stored on the microprocessor (1200,FIG. 1), the camera (1100, FIG. 1) feeds real-time images to themicroprocessor. By employing unique computer comparison algorithmsdiscussed in greater detail below, the microprocessor 1200 compares theshape, size, and color of the objects in the real-time camera images tothe reference images. In an alternate embodiment, image entropy, edgedetection, and face detection algorithms might also be incorporated intothe comparison protocols of the reference image and the incongruity.Finally, if a match is found, the object in the real-time image isdesignated as the animal of the type corresponding to the referenceimage and categorized. When a positive match occurs with one of thereference images, additional processing will occur, including sending awireless signal to alert the user that an animal of interest has beenspotted at the area of interest. This alert is designated as an informedaction. Whenever a positive match does not occur with a reference image,no message or alert will be sent to the user and the image is stored ordeleted. This alert is an uninformed action.

FIG. 3 presents a detailed flowchart of the computer vision basedalgorithm. In the algorithm, the first step is to learn the empty/staticbackground image or appearance of the area of interest. Suitably, thestatic or empty scene reference image is the theoretical image of thearea without any moving objects. In one embodiment, the system learnsthe background as follows: a video or photo stream is delivered to thesystem while all animal detection and tracking is disabled; next, thesystem watches the scene and measures the frame-to-frame foreground, ascalculated from a simple frame subtraction wherein the subtraction isdone in rectangular subsections of the scene, which has been portionedinto a grid of configurable size. Each subsection of the grid iscompared to the corresponding subsection in the next frame of the photoor video stream, and when that section of the grid has “settled”(meaning very little motion), that subsection is considered complete andthe image in that subsection is stored as the static background for thatsubsection; finally, as soon as all subsections of the grid have acompleted static image, the full static background is stitched togetherfrom all the background subsections and stored in the system. Insummary, the background learning algorithm (1) splits the frames of aphoto or video stream into a grid, (2) processes the photo or videostream for zero motion in each grid section, (3) stores each still gridsection, and (4) stiches a static background together with the stills ofeach grid section. For non-volatile (e.g. areas of little weather oractivity), the static background is learned quickly, for volatile sceneswith a lot of motion, the static background may take time to learn.Suitably, configurable thresholds for learning time, grid size, etc. maybe set to the system. The background learning algorithm suitablyprevents the need for the user to “set the background”, although, in adifferent embodiment, a simple image of the background without anyanimals may be used to set the static image. In that case, to accountfor real-life situations that involve non-static areas due to wind andother factors, a filtering technique may be employed to account for anysmall variations in images of the static or empty area of interest.

With the reference image of the static area of interest loaded on themicroprocessor, real-time images of the area of interest are loaded onthe microprocessor from the camera wherein overlapping aspects of thereference image and real-time image are subtracted from each other toidentify incongruities between the images. FIGS. 4A through 4Dillustrate this process. As shown in those figures, the reference image1000 is compared with a real time image 2000. Preferably, overlappingaspects of the reference image and real-time image are subtracted fromeach other to identify incongruities 3000 between the images. In FIGS.4A through 4D, incongruities 3000 are identified in the form of deer, adog, a horse, and birds.

If any incongruities are identified, then the incongruity is comparedwith the reference images of animals to determine whether theincongruity matches the reference images (e.g., a previously identifiedbuck or doe). This process is illustrated by FIG. 5, which compares theincongruities 3000 of FIGS. 4A through 4D with a reference image of abuck 4000. As shown, only the reference incongruity of FIG. 4A matcheswith the reference image. When a match occurs, the scene image is savedto a gridded images folder and becomes a future reference image. When amatch occurs an alert is sent to a user regarding the match.

In one embodiment, the system may be used to track specific animals orentities that frequent the area of interest. In this embodiment, if theobject has not been previously identified, the object is comparedagainst the stored reference images to determine if it is a new matchwith respect to the reference images (e.g., the object is a new buck ordoe in the scene). When new animals are detected at an area of interest,an alert is delivered via wireless communication and the scene imagewith the new animal is stored in the “new” categorized images folder.When a reference detects an incongruity with the empty/static backgroundimage that cannot be matched with a reference image, the scene image issaved to the unknown images folder. If no objects are detected, thebackground learning algorithm may be employed to reset the static stateimage.

Example 1 Distinguishing a Doe v. Buck

In one example of a typical use, the system may be used to alert ahunter to the presence of a buck, but not does, in a particular area ofinterest. First, the camera, microprocessor, and wireless communicationdevice can be established in the area of interest. Second, a staticimage of the area of interest may be loaded into or learned by themicroprocessor. Next, reference images of bucks and does can be loadedinto the microprocessor. After setup, the camera may begin sendingreal-time images or video to the microprocessor wherein the images arecompared programmatically to the static image of the area. Ifincongruities exist between the static image and the real-time image,then the size, shape and color of the incongruities are programmaticallycompared to the size, shape, and color of the bucks and does in thereference images. When a match occurs between the incongruity and thereference images for bucks, an alert is sent to the user that a buck isin the area.

Example 2 Distinguishing a Buck v. Horse

In one example of a typical use, the system may be used to alert ahunter to the presence of a buck, but not horses or other farm animals,in a particular area of interest. First, the camera, microprocessor, andwireless communication device can be established in the area ofinterest. Second, a static image of the area of interest may be loadedinto the microprocessor or learned by the system. Next, reference imagesof bucks and horses can be loaded into the microprocessor. After setup,the camera may begin sending real-time images or video to themicroprocessor wherein the images are compared programmatically to thestatic image of the area. If incongruities exist between the staticimage and the real-time image, then the size, shape and color of theincongruities are programmatically compared to the size, shape, andcolor of the bucks and horses in the reference images. When a matchoccurs between the incongruity and the reference images for bucks, analert is sent to the user that a buck is in the area.

Example 3 Distinguishing a Particular Prize Buck v. A Young Buck

In another example of a typical use, the system may be used to alert ahunter to the presence of a prize buck, but not young bucks, in aparticular area of interest. First, the camera, microprocessor, andwireless communication device can be established in the area ofinterest. Second, a static image of the area of interest may be loadedinto the microprocessor or learned by the system. Next, reference imagesof a particular prize buck and regular bucks and can be loaded into themicroprocessor. After setup, the camera may begin sending real-timeimages or video to the microprocessor wherein the images are comparedprogrammatically to the static image of the area. If incongruities existbetween the static image and the real-time image, then the size, shapeand color of the incongruities are programmatically compared to thesize, shape, and color of the Prize bucks and young bucks in thereference images. When a match occurs between the incongruity and thereference images for the prize buck, an alert is sent to the user thatthe particular prize buck is in the area.

Other features will be understood with reference to the drawings. Whilevarious embodiments of the method and apparatus have been describedabove, it should be understood that they have been presented by way ofexample only, and not of limitation. Likewise, the various diagramsmight depict an example of an architectural or other configuration forthe disclosed method and apparatus, which is done to aid inunderstanding the features and functionality that might be included inthe method and apparatus. The disclosed method and apparatus is notrestricted to the illustrated example architectures or configurations,but the desired features might be implemented using a variety ofalternative architectures and configurations. Indeed, it will beapparent to one of skill in the art how alternative functional, logicalor physical partitioning and configurations might be implemented toimplement the desired features of the disclosed method and apparatus.Also, a multitude of different constituent module names other than thosedepicted herein might be applied to the various partitions.Additionally, with regard to flow diagrams, operational descriptions andmethod claims, the order in which the steps are presented herein shallnot mandate that various embodiments be implemented to perform therecited functionality in the same order unless the context dictatesotherwise.

Although the method and apparatus is described above in terms of variousexemplary embodiments and implementations, it should be understood thatthe various features, aspects and functionality described in one or moreof the individual embodiments are not limited in their applicability tothe particular embodiment with which they are described, but insteadmight be applied, alone or in various combinations, to one or more ofthe other embodiments of the disclosed method and apparatus, whether ornot such embodiments are described and whether or not such features arepresented as being a part of a described embodiment. Thus the breadthand scope of the claimed invention should not be limited by any of theabove-described embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open-ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like, the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof, the terms “a” or“an” should be read as meaning “at least one,” “one or more,” or thelike, and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that mightbe available or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases might be absent. The use ofthe term “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, might be combined ina single package or separately maintained and might further bedistributed across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives might be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

The claims as originally filed are incorporated by reference in theftentirety.

We claim:
 1. A system for (a) monitoring real-time animal activity in anarea of interest using computer aided image comparison of a real-timeimage of the area and reference images of animals of interest and (b)providing informed alerts to a user regarding animal type in said area.2. The system of claim 1 wherein the system comprises: a camera mountedin a remote area of interest; a computer operated microprocessor that isconnected to the camera; wherein, the microprocessor is loaded withreference images of animals; wherein, the camera is configured tocontinuously feed real time images of the remote area of interest to themicroprocessor; wherein the microprocessor is configured to (a)construct a static reference background image from said continuous feedof real time images, to (b) programatically compare the continuous feedof real-time images of the area to the static reference background imageto identify any incongruity in the real-time images, and to (c) comparesaid reference images of animals of interest to the incongruities todetermine whether any incongruity in the real time image is an animal ofinterest.
 3. The system of claim 2, wherein the microprocessor isconfigured with programming language and wireless communication hardwareso that: if, after comparison, the incongruities have a match with thereference images, then a wireless signal alerts a user and, if thedetected animal is relevant, the user is notified of the animal'spresence and activity in the area.
 4. The system of claim 2, wherein thecamera is connected wirelessly to the computer operated microprocessor.5. A system comprising: a camera mounted in a remote area; amicroprocessor connected to the camera; a wireless communication unitconnected to the microprocessor; wherein the camera is configured tocapture images or videos of the area of interest and load those imagesto the microprocessor; wherein, the microprocessor (a) is pre-loadedwith references images of animals of interest, (b) is continuallylearning the appearance or background of the area in a static state by aconstant photo or video feed from the camera, and (c) is configured tocompare the images or video images taken by the camera with thereference images to determine the type, size, and color of animalsentering the area of interest; and, wherein the wireless communicationunit is configured for sending alerts regarding the animal'sinformation.
 6. The system of claim 5 wherein the microprocessorfeatures computer code in the form of computer image comparisonalgorithms for comparing images or video from the camera with referenceimages of animals of interest.
 7. The system of claim 6 wherein thecomputer code comprises a high level logic flowchart as follows: (1) animage of the static state background or appearance of the area islearned; (2) reference images of animals of interest and a static imageof the area of interest may be input and stored in the microprocessor;(3) with the reference images stored on the microprocessor, the camerafeeds real-time images to the microprocessor; (4) if a match is found,the object in the real-time image is designated as the animal of thetype corresponding to the reference image and categorized, wherein whena positive match occurs with one of the reference images, additionalprocessing will occur, including sending a wireless signal to alert theuser that an animal of interest has been spotted at the area ofinterest.
 8. The system of claim 6 wherein learning the staticbackground image or appearance of the area of interest is accomplishedvia (a) delivering a video or photo stream is delivered by the camera tothe microprocessor, (b) the microprocessor measures the frame-to-frameforeground, as calculated from a simple frame subtraction wherein thesubtraction is done in rectangular subsections of the scene, which hasbeen portioned into a grid of configurable size so that each subsectionof the grid is compared to the corresponding subsection in the nextframe of the photo or video stream, and when that section of the gridhas very little variance, that subsection is considered complete and theimage in that subsection is stored as the static background for thatsubsection, and (c), when all subsections of the grid have a completedstatic image, the full static background is stitched together from allthe background subsections and stored in the microprocessor.
 9. Thesystem of claim 6 wherein the microprocessor is configured to learn thestatic background image of the area of interest via (1) splitting theframes of a photo or video stream into a grid, (2) processing the photoor video stream for minor variance in each grid section, (3) storingeach grid section with minor variance, and (4) stitching a staticbackground together with the stored grid sections with minor variance.10. The system of claim 8 wherein the real-time images of the area ofinterest that are loaded on the microprocessor from the camera arecompared so that overlapping aspects of the reference image andreal-time image are subtracted from each other to identify incongruitiesbetween the images and wherein if any incongruities are identified, thenthe incongruity is compared with the reference images of animals todetermine whether the incongruity matches the reference images.
 11. Thesystem of claim 10 wherein the microprocessor is a smartphone.